Rule extraction from local cluster neural nets

This paper describes RULEX, a technique for providing an explanation component for Local Cluster (LC) neural networks. RULEX extracts symbolic rules from the weights of a trained LC net. LC nets are a special class of multilayer perceptrons that use sigmoid functions to generate localised functions. LC nets are well suited to both function approximation and discrete classification tasks. The restricted LC net is constrained in such a way that the local functions are ‘axis parallel’ thus facilitating rule extraction. This paper presents results for the LC net on a wide variety of benchmark problems and shows that RULEX produces comprehensible, accurate rules that exhibit a high degree of fidelity with the LC network from which they were extracted. Introduction In [1] Geva et.al. describe the Local Cluster (LC) network, a sigmoidal perceptron with 2 hidden layers where the connections are restricted in such a way that clusters of sigmoids form local response functions similar to Radial Basis Functions (RBFs). They give a construction and training method for LC networks and show that these networks (i) exceed the function representation capability of generalised Gaussian networks, and (ii) are suitable for discrete classification. They also describe a restricted version of the LC network and state that this version of the network is suitable for rule extraction without however describing how this is possible. Local function networks are attractive for rule extraction for two reasons. Firstly, it is conceptually easy to see how the weights of a local response unit can be converted to a symbolic rule. Local function units are hyper-ellipsoids in input space and can be described in terms of a reference vector that represents the centre of the hyper-ellipsoid and a set of radii that determine the effective range of the hyper-ellipsoid in each input dimension. The rule derived from the local function unit is formed by the conjunct of these effective ranges in each dimension. Rules extracted from each local function unit are thus propositional and of the form: IF æ 1 # i # n : xi 0 [ xi lower , xi upper ] THEN pattern belongs to the target class ...(1) where [ xi lower , xi upper ] represents the effective range in the ith input dimension. Secondly, because each local function unit can be described by the conjunct of ranges of values in each input dimension it makes it easy to add units to the network during training such that the added unit has a meaning that is directly related to the problem domain. In networks that employ incremental learning schemes a new unit is added when there is no significant improvement in the global error. The unit is chosen such that its reference